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Abstract: We study the neutral Higgs boson production via the gluon fusion process with 
the t + t~ final state at the upgraded Fermilab Tevatron, including a complete simulation of 
signal channels and leading background processes. For the SM Higgs boson, this h — > t + t~ 
channel may provide important addition for the Higgs boson discovery in the mass range 
120 — 140 GeV. In minimal supersymmetric models, natural enhancement for the signal 
rate over the SM expectation makes the h,H,A — * t + t~ signal observable for large tan f3 
and low Ma, which may lead to full coverage for SUSY Higgs parameters at the Tevatron 
with a moderate integrated luminosity. 
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Standard Mode]. 



It has been extensively discussed to what extent the Higgs bosons can be discovered 
at the upgraded Tevatron ||]. The leading contribution to the Higgs boson production 
at hadron colliders comes from gluon fusion via heavy quark loops, with a typical cross 
section of one pb at the Tevatron energies. However, a light Higgs boson mainly decays 
into bb and the huge QCD background precludes any hope for finding the Higgs boson 
in this channel. That is the reason why the most favorable process for finding the Higgs 
boson at the Tevatron up to 130 GeV is the associated production W, Z + h 0, with a 
cross section of the order of 0.5 pb. On the other hand, for a SM Higgs boson of mass 140 
GeV < rtih < 190 GeV, it may be possible to observe the mode h —> WW* f| if higher 
luminosity of order 10 — 30 fb _1 becomes available. 

In this paper we study the feasibility of utilizing the gluon fusion process but with the 
t + t~ final state 

PP -> 99 -> h -> t + t~. (1) 
There are several motivations to study this channel. First, it is very difficult at the Tevatron 
to find a SM Higgs boson signal in the mass range 130 — 140 GeV at the interplay between 
bb and WW* final states. It would be desirable to find other potentially useful channels 
§ and the r + r~ mode is a natural candidate to consider since it has a branching ratio of 
about 10% in the mass range of interest. Second, this mass range of the current interest 
is the probable region for in Supersymmetric models (MSSM) and one may expect 
some possible enhancement for process Eq. (|l|) due to sparticle loops [||, ||, [?J. Thirdly, 
it is especially important to determine the relative coupling strength of /it + t~ and hbb 
since many new physics scenarios predict different relations of these Yukawa couplings ||. 
Finally, since r leptons are a prominent signal for various models of new physics, the final 
state containing r s has been considered in many recent studies and their experimental 
identification has become better understood [10]. We find that the channel of Eq. (|l|) may 



provide important addition for the SM Higgs boson discovery in the mass range 120 — 140 
GeV. Especially in the MSSM, significant improvement at low Ma and high tan (3 may lead 
to full coverage for SUSY Higgs parameters at the Tevatron. 

In order to perform a complete signal and background simulation, we use the event 
generator PYTHIA v6.134 jHJ for both signal and backgrounds. The effects of initial 
and final state radiation (IFSR) and hadronization have been taken into account. Final 
state decays of the polarized r leptons have been treated properly by making use of the 



package TAUOLA v2.6 pj. CTEQ4M parton distributions pj have been used. The QCD 
scale Q for both the factorization and renormalization is set to the average of transverse 
momenta of the outgoing particles at the parton level. Because of the limited statistics, 
we consider all possible r-decay modes, hadronic and leptonic. Detector parameters and 
energy resolution were chosen the same as in simulation for the Run II Workshop |IJ. In 
particular, the jet energies are smeared according to a Gaussian spread 

AE 0.8 



E y/E GeV ' 

Due to the missing energy carried away by the two neutrinos v T v T in the final state, we 
can only effectively reconstruct the Higgs boson mass from the decay products of t + t~ if 



- 1 - 



the tau pairs are not back-to-back in the transverse plane [ 14 1 . For this reason we need to 
consider the Higgs boson production with a finite transverse momentum (pj>). We adopt 
the process gg(q) — » /ig((/)+initial-final state radiation (IFSR) for the signal simulation as 
implemented in [|TTJ. The cross section of this process is of the order of 0.3 pb for rri} l = 120 
GeV with the cut p^ > 20 GeV. With the branching fraction for h — > t + t~ of roughly 
10%, one obtains cross sections for p ] T > 20 GeV at yfs = 2 TeV 

a(pp hj -» r + T~j) = 44, 28, 15 fb for m h = 120, 130, 140 GeV, (2) 

respectively. 

The leading irreducible background to the signal is pp — > Zj — > t + t~ j. The other 
reducible but huge background comes from QCD jets that can fake a r final state, pp — > 
333 ~^ T + T~j. Using the acceptance cuts on p J T , pseudo-rapidity and separation of the 
final state jets 

p^>20GeV, |r/j|<2, ARjj > 0.5, (3) 
the cross sections for the backgrounds are 

a(pp -> Zj r + T-j) = 7 x 10 4 fb, (4) 
°(VV - ij'i) = 2.5 x 10 8 fb. (5) 

The overwhelming background is formidable and a more efficient set of kinematical cuts is 
necessary in order to have a chance of extracting the signal. 

Although less important, there are other backgrounds that we should comment on. 
First, the final state W^jj with — > e/fi/r and j — > r can constitute a background to 
the signal. The production rate with the jet cuts of Eq. @ is a (W ± jj) = 300 pb. With W 
leptonic decay and a jet faking a r, this background rate becomes about 200 fb. The next 
potentially sizeable background is from W + W~j production. This background has a rate 
2.0 pb. With W leptonic decay it becomes a(W + W~j -» £ + £~j) = 220 fb (£ = e,/i,r). 
These background rates of Wjj and WWj are still somewhat larger than the signal rate. 
However, since those backgrounds are continuous ones, they will not be important after 
the judicial cuts and especially after the Higgs mass reconstruction, as we will discuss next. 

To unambiguously identify the Higgs boson signal, one must reconstruct the mass peak 
M TT at m/j. This is also the most efficient way to discriminate against the backgrounds. 
Due to the fact that the t + t~ from Higgs decay are ultra-relativistic, the jet (or lepton) 



and the neutrino(s) from the r decay are essentially collinear along p T [14]. We can thus 
solve for the two neutrino momenta as long as the t + t~ are not collinear. Alternatively, 
one could consider to make use of the cluster transverse mass variable Mj r gj|. This 
transverse mass variable should reach maximum near m/j,, but has a broad tail below mh- 
Depending on the decay modes, events with t + t~ j signature lead to final states 
such as: the pencil-like two r-jets +j; or one pencil-like T-jet+ lepton +j; or two leptons + 
j. All decay channels have at least two missing neutrinos v T v T , and each charged lepton I 
will be accompanied by another neutrino Among those channels, the leptonic channels 
lead to a better energy-momentum determination but have a smaller branching fraction. 
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On the other hand, the hadronic channels have a higher rate but a poor energy resolution 
for mass reconstruction. 

In Fig. | we show the normalized reconstructed mass distributions of M TT for the signal 
and backgrounds for = 120 GeV. Figures |l|(a) is for both t + t~ decaying hadronically 
and Fig. 0(b) for t + t~ decaying leptonically. Although the jjj background rate is a 
lot higher than that of Zj to begin with, the reconstructed mass spectra are sufficiently 
different from the peak structure of the signal. This feature is also true for other faked 
backgrounds Wjj and WW j. In contrast, the Zj background naturally presents a peak 
at the Z mass and leads to a long tail after Mz- This constitutes the major irreducible 
background as we will see later. This becomes the limiting factor for us to explore a Higgs 
boson below and near 110 GeV. As anticipated, the leptonic channels have sharper signal 
mass peaks. Taking into account the kinematical features of the signal and backgrounds 
discussed above, we devise the following set of kinematical cuts. We first require the r 
identification for the hadronic (tj) or leptonic (r^) decay, Cut I: 

tj : • p> T > 15 GeV, \rf\ < 2 

• one or three tracks in 10° cone with no additional tracks in 30° cone 

• the invariant mass of tracks is less than 2 GeV 
T £ : • p l T > 10 GeV \rf \ < 2; 

• no additional tracks in 30° cone for jjj mode : fyr > 20 GeV, 

where the missing transverse energy fyp is defined by the imbalance of the observed particles, 
and it is also smeared according to Gaussian distribution with standard deviation 0.5\£fr. 
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Figure 1: Normalized reconstructed mass M TT distributions for t + t _ decaying (a) hadronically 
and (b) leptonically. The solid lines are for mh = 120 GeV, the dashed lines for the Zj background 
and the dotted lines for jjj background. 
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Thetfr cut is to help triggering the jjj events. For the jj channel we require 2rj with no 
isolated electrons or muons (defined by r^), for the j£ channel we require only one Tj and 
one isolated lepton, and for the U channel we require no Tj in addition to the two isolated 
leptons. 

In Cut I, the cut on the jet invariant mass is essential for reducing the huge QCD 
background in which a QCD parton jet fakes a r. The efficiency of r ID from the Z 
decay is in agreement with more realistic studies [10|. The efficiency of more energetic t's 



from Higgs decay has not been previously studied for the Tevatron with the full detector 
simulation. We find that the efficiency of r ID is quite px and process-dependent, typically 
being 60 - 70% for Tj and 50 - 60% for n for m h = 120 - 140 GeV. After the r-pair 
identification, we then impose the next level of refined kinematical cuts, Cut II: 

• pjt > m h /6, > m h /9, 

• additional jet (not Tj) with pL > m^/6 and {rfl < 4. 

Our final set of cuts defines the optimal windows in M TT and M^ T that will be used 
to confirm the existence of a signal as well as to estimate the significance for the signal 
observation, Cut III(A): 

m h - 0.5AM < M TT <m h + 3AM, 
m h - 4AM < Mj r < m h + 3AM, 

for pure hadronic decay of both r's and Cut III(B): 

m h - AM < M TT <m h + 3AM, 
m h - 4.5AM < Mj T <m h + 3AM, 



for others channels of r decay, where AM = w m^/GeV GeV. 

In the case of the QCD faked background to r, we do not directly go through the 
procedure outlined in Cut I as for the signal. Instead, we use the probability for a jet 
to be misidentified to be 0.5% g, and n to be 0.01%. We then apply Cuts II 

and III. Since the probability for QCD background to fake two leptons is very small, the 
background to the di-lepton channels is negligible in comparison with the signal, especially 
with the irreducible Zj background. 

With the substantial efforts discussed above, we have effectively suppressed the back- 
grounds with respect to the signal. The signal efficiencies are at a percentage level. Table 
[l] shows the final number of events for all signal channels and their corresponding back- 
grounds at the Tevatron with an integrated luminosity 10 fb _1 . Also shown in Table || are 
the signal-to-background ratios (S/B). Because of the overwhelming QCD background, the 
S/B ratio is the highest for 11 channel of r decay (~ 6 — 8%), intermediate for j£ channel 
(~ 4 — 5%), and the lowest for jj channel (~ 3%). With respect to S/VB and S/B ratios, 
the jl channel is probably the best one for signal identification. Nevertheless, the rather 
small S/B ratios render the signal observation systematically challenging. 

Our main results, the luminosities required for a 95% CL exclusion or a 5a discovery 
of the SM Higgs boson via h t+t~ at -/s = 2 TeV for m h = 120, 130 and 140 GeV, 
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Table 1: Final number of events of signal for all channels from h — > t + t for representative Higgs 
boson masses, corresponding backgrounds at y/s = 2 TeV per 10 fb _1 integrated luminosity and 
S/B ratio. 



are summarized in Table A 95% CL exclusion limit for the SM Higgs boson in the mass 
range 120 — 140 GeV via this single channel h — > t + t~ would require a total integrated 
luminosity of 14 — 32 fb . To gain an idea on the signal observation in theories beyond 
the SM, also given in Table ^ are the necessary enhancement factors k over the SM rate 
{pnew = k&sm) for reaching 95% and 5c signal with 2 and 15 fb -1 luminosity. 
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Table 2: Integrated luminosities needed to reach a 95% CL exclusion, 3 and 5cr discovery for a 
SM Higgs boson at the Tevatron, and the enhancement factor re (at 2 and 15 fb -1 ) needed to reach 
a 95% CL exclusion and 5a discovery. 

The t + t~ channel can provide new addition in combination with the other promising 
channels such as Wh, Zh and h -> WW |, |, § to improve the overall observability of the 
SM Higgs boson. To illustrate the potential improvement, we estimate the total integrated 
luminosities needed for the 95%, 3a, 5a effects for = 120 — 140 GeV by combining the 
Run II report and our h — > t + t~ results, according to a relation 

L 1 = 1 + L*2 i 

as shown in Table ||. We have followed the convention from the Run II report that the 
numbers in Table || correspond to the delivered machine luminosity; while the significance 
values have been evaluated by combining CDF and DO (doubling the delivered luminosity). 
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Especially for the most difficult region ~ 140 GeV, the luminosity needed may be 



reduced by about 40% with the addition of the h 
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Table 3: Integrated luminosities needed (for one of the 
two experiments) to reach a 95% CL exclusion, 3 and 5tr 
discovery for a SM Higgs boson by combining the cur- 
rent t + t~ results and the Run II report jl). Numbers in 
parentheses are from Q. 



t + t channel. 

In the minimal supersymmetric 
models, there are two more neutral 
Higgs bosons H, A to contribute to 
the t + t~ mode. In addition, there 
are new squark-loop contributions to 
Higgs boson production through gluon 
fusion. In our analyses, we use HIGLU 



program [16] for the calculation of 
the cross section of Higgs produc- 
tion process in the MSSM. For sim- 
plicity, we fix the squark masses to 1 TeV. We find that left-right squark mixing is not a 
crucial factor in our analysis. This simplification allows us to illustrate the potential of the 
Tevatron search by the Higgs sector parameters Ma — tan (3 in the conventional way in a 
two dimensional plane. In Fig. |2|(a) we plot contours of the enhancement factor over the SM 
expectation which is defined to be the rate at m^ M = Ma- The large enhancement above 




Figure 2: (a) Contours (above the curves) in (Ma, tan (3) plane for the enhancement factor above 
the SM Higgs boson signal rate as labeled. The contributing Higgs states are also indicated by 
A, h, H separated by the dashed lines; (b) contours (above the curves) for 95% CL exclusion and 
5cr discovery for pp — > (A, h) + j — * t + t~ j process within MSSM at the Tevatron with 2 and 15 
lb : . 
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the SM mainly comes from the new channels from H, A. The contributing Higgs states are 
also indicated by the dashed lines in Fig. ||(a) with the explicit labels A, h, H. Also, large 
tan/3 will enhance the effects from bottom/sbottom loops. On the other hand, the decay 
branching fractions to t + t~ may be enhanced by about 30% at most for various choices of 
the SUSY parameters. For Ma ~ 150 — 180 GeV, H,A^ t + t~ is still observable; while 
the SM mode dies away due to the opening of the dominant WW, ZZ channels. When 
combing the cross sections from channels of A and h production, we have used Gaussian 
combination criteria. In Fig. ||(b) we show the 95% CL exclusion and 5a discovery contours 



in MSSM for the t + t~ mode at the Tevatron. The direct searches at LEP2 [17] have put a 
lower bound on the parameters as indicated by the dotted curve. The shaded region is the 
uncovered 5a hole at Run II with 15 fb _1 jj], |7j without considering the t + t~ mode. We 
see that the addition of the t + t~ mode could reach 5a (2a) full coverage for SUSY Higgs 
parameters with 15 fb _1 (2 fb _1 ). As a final remark, we note that in some region of the 
parameter space hbb Yukawa coupling is accidentally suppressed due to radiative effects. 
This could lead to a substantial increase of the h — > rf branching fraction ||. However, 
we found that this gain in branching fraction is mostly balanced by the reduction in the 
Higgs production via the hbb coupling. 

In summary, we have studied the potential of the upgraded Tevatron for searches of 
the neutral Higgs boson in the SM and in the MSSM via the rr + j channel. We found 
that with this single channel alone, the total integrated luminosity needed for a 95% CL 
exclusion is 14 — 32 fb _1 for = 120 — 140 GeV. The addition of the h — > t + t~ channel 
could improve the SM Higgs observation, as presented in Table ^. For the most difficult 
region ~ 140 GeV, the luminosity needed for a 5a signal could be reduced by about 
40%. However, we must note that due to the rather small signal-to-background ratios, the 
search for t + t~ channels is systematically challenging in the Standard Model case. 

For the case of MSSM, the signal cross sections can be enhanced by a significant factor 
due to the addition of H, A, and due to high tan (3 or squark-loop contributions. We found 
5a (2a) full coverage for the SUSY Higgs parameters with 15 fb _1 (2 fb _1 ) by including the 
r + r~ channel. Our analyses is applicable for other generic neutral scalar or pseudo-scalar 
via gluon fusion production and with substantial branching fraction to t + t~. More details 



of our analyses will be presented in an extended version of this work [18]. 
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